Skip to content

[TPDE][LLVM] Fix register exhaustion for register-heavy encoding functions - #46

Open
marcauberer wants to merge 1 commit into
tpde2:masterfrom
marcauberer:claude/github-issue-45-fix-61dmps
Open

[TPDE][LLVM] Fix register exhaustion for register-heavy encoding functions#46
marcauberer wants to merge 1 commit into
tpde2:masterfrom
marcauberer:claude/github-issue-45-fix-61dmps

Conversation

@marcauberer

Copy link
Copy Markdown
Contributor

Some tpde_encodegen encoding functions (e.g. encode_of_mul_i128, the __int128 multiply-with-overflow snippet) need up to 10 GP registers live at once. With 5 GP registers permanently held by cross-block fixed assignments, only 9 of the 14 allocatable GP registers remained, so the register allocator aborted with "ran out of registers for scratch registers" instead of compiling. Reduce
NUM_FIXED_ASSIGNMENTS[GP_BANK] from 5 to 4 to leave enough headroom.

Also add a two-layer assertion mechanism to catch a regression of this class of bug earlier:

  • A static_assert ties NUM_FIXED_ASSIGNMENTS[GP_BANK] to the documented worst-case concurrent register demand of a known encoding function, failing the build instead of crashing at runtime if the headroom is ever reduced again.
  • The two sites that can hit "ran out of registers for scratch registers" now log the bank id and allocatable register count before aborting.

Fixes #45

…tions

Some tpde_encodegen encoding functions (e.g. encode_of_mul_i128, the
__int128 multiply-with-overflow snippet) need up to 10 GP registers
live at once. With 5 GP registers permanently held by cross-block
fixed assignments, only 9 of the 14 allocatable GP registers remained,
so the register allocator aborted with "ran out of registers for
scratch registers" instead of compiling. Reduce
NUM_FIXED_ASSIGNMENTS[GP_BANK] from 5 to 4 to leave enough headroom.

Also add a two-layer assertion mechanism to catch a regression of this
class of bug earlier:
- A static_assert ties NUM_FIXED_ASSIGNMENTS[GP_BANK] to the documented
  worst-case concurrent register demand of a known encoding function,
  failing the build instead of crashing at runtime if the headroom is
  ever reduced again.
- The two sites that can hit "ran out of registers for scratch
  registers" now log the bank id and allocatable register count before
  aborting.

Fixes tpde2#45

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: marcauberer <contact@marc-auberer.com>
Claude-Session: https://claude.ai/code/session_016Gygc35eVJErYtDXb7pedF
@T0b1-iOS

T0b1-iOS commented Aug 5, 2026

Copy link
Copy Markdown
Member

So, from a gut feeling I'd like to avoid pessemizing all the code we generate because we run out of register in (what I assume to be) a corner case. What we would rather think about is to scrap the encode function for signed 128 multiplication with overflow and replace it by a call to an out-of-line function. We used to do that before encodegen existed but I presume we stopped that since we could generate it with a template and it just worked :D

Since there is no implementation for 128 bit multiplication with overflow in gcc's runtime we'd have to emit the function ourselves but that should not be a huge problem.

Would that also be an acceptable solution for your use-case?
In any case I would first measure the impact of using 4 fixed regs on performance when I have some time and if it really doesn't make a difference we could also consider doing this I presume. Did you do any performance testing of this change?

@marcauberer

Copy link
Copy Markdown
Contributor Author

I share this gut feeling, this was only the quick fix from our side, so feel free to suggest/follow other approaches.

We did run our own frontend performance suite and don't saw any regressions. Based on experience, I would not expect much of an regression from a reduction from 5 to 4 GPRs.

If I understand you correctly, the approach to generate a separate function and call it was used in the past, but currently is not part of the codebase?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fatal Error in register allocation with llvm.smul.with.overflow.i128 intrinsic and 5 fixed registers

3 participants